Revert "images: Pin freeipa container to an older version" by Venefilyn · Pull Request #9260 · cockpit-project/bots

Venefilyn · 2026-07-08T16:47:38Z

This reverts commit 947a092.

See cockpit-project/cockpit#21564
and https://access.redhat.com/errata/RHBA-2025:7246

This is no longer an issue in newer c9s tags.

Also adds a DNS server to podman-run of FreeIPA

Needed since last year when image was updated and had a breaking change
with how they approach resolved.

Venefilyn · 2026-07-08T16:47:50Z

/image-refresh services

cockpituous · 2026-07-08T16:48:35Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://api.dev.testing-farm.io/v0.1/requests/1f2b7a69-5809-4757-9812-59ec2e3098d6

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "53315ddd86d955f81d82815d8a1d810cc9af0167",
  "pull": 9260,
  "slug": "image-refresh-services-53315ddd-20260708-164804",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-08T17:35:54Z

image-refresh services started on c3524d76835d. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-53315ddd-20260708-164804/log.html

Venefilyn · 2026-07-08T17:55:56Z

Checking the bots/vm-run services and running a modified /root/run-freeipa where --dns=127.0.0.1 is added and image changed to latest c9s version. I still get issues though

podman run -d --rm --name freeipa -ti -h f0.cockpit.lan     -e IPA_SERVER_IP=10.111.112.100     --dns=127.0.0.1     -p 10.111.112.100:53:53/udp -p 10.111.112.100:53:53 -p 80:80 -p 443:443 -p 389:389 -p 636:636 -p 88:88 -p 464:464 -p 88:88/udp -p 464:464/udp -p 123:123/udp     -v /var/lib/ipa-data:/data:Z     -v /sys/fs/cgroup:/sys/fs/cgroup:ro     quay.io/freeipa/freeipa-server:centos-9-stream     -U -p foobarfoo -a foobarfoo -n cockpit.lan -r COCKPIT.LAN --setup-dns --no-forwarders --no-ntp

systemd 252-71.el9 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
Detected virtualization container-other.
Detected architecture x86-64.
Queued start job for default target Minimal target for containerized FreeIPA server.
Wed Jul  8 17:30:31 UTC 2026 /usr/sbin/ipa-server-configure-first upgrade
FreeIPA server is already configured, starting the services.
[  124.782831] Out of memory: Killed process 1835 (ipactl) total-vm:170468kB, anon-rss:69716kB, file-rss:12kB, shmem-rss:0kB, UID:0 pgtables:324kB oom_score_adj:0
[  125.353974] Out of memory: Killed process 1861 (ipa-server-guar) total-vm:54228kB, anon-rss:31084kB, file-rss:52kB, shmem-rss:0kB, UID:0 pgtables:148kB oom_score_adj:0
Wed Jul  8 17:30:40 UTC 2026 /usr/sbin/ipa-server-configure-first update-self-ip-address
Unable to resolve "f0.cockpit.lan". Is --dns=127.0.0.1 set for the container?

cockpituous · 2026-07-08T18:03:57Z

image-refresh services succeeded: https://github.com/cockpit-project/bots/commits/image-refresh-services-20260708-180354

Venefilyn · 2026-07-08T18:09:14Z

/image-refresh services

cockpituous · 2026-07-08T18:09:58Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://api.dev.testing-farm.io/v0.1/requests/7c4e29f1-8253-4d35-8cb2-94d25fad15d9

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "1d15d558d719afe5735f6a060ae155ba9c56d602",
  "pull": 9260,
  "slug": "image-refresh-services-1d15d558-20260708-180927",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-08T18:51:44Z

image-refresh services started on a9d444fb21db. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-1d15d558-20260708-180927/log.html

cockpituous · 2026-07-08T19:22:40Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn · 2026-07-08T19:34:56Z

/image-refresh services

cockpituous · 2026-07-08T19:35:38Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://api.dev.testing-farm.io/v0.1/requests/a73aba5a-dd02-4e2e-9d28-8877774c0ce8

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "1d15d558d719afe5735f6a060ae155ba9c56d602",
  "pull": 9260,
  "slug": "image-refresh-services-1d15d558-20260708-193507",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-08T20:08:49Z

image-refresh services started on 2977375e114c. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-1d15d558-20260708-193507/log.html

cockpituous · 2026-07-08T20:35:38Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn · 2026-07-08T23:50:24Z

@allisonkarlitskaya Any idea? 👀

Try contacting this VM's SSH server via 'ssh vsock%4294967295' from host.

services login:
---- Console log ends here ----
image-create: Waiting for machine poweroff timed out
Creating machine without build capabilities based on the image type

# image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.
# Duration: 1608.1784431934357s
Transient error during GitHub request, attempt #0: Remote end closed connection without response
Traceback (most recent call last):
  File "/work/make-checkout-workdir/./image-refresh", line 142, in <module>
    main()
    ~~~~^^
  File "/work/make-checkout-workdir/./image-refresh", line 127, in main
    branch = image_refresh(github, args.image, args.issue, verbose=args.verbose, dry_run=args.dry_run)
  File "/work/make-checkout-workdir/./image-refresh", line 58, in image_refresh
    run(
    ~~~^
        './image-create',
        ^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        verbose=verbose,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/work/make-checkout-workdir/./image-refresh", line 32, in run
    result = subprocess.run(cmd, check=check, env=env)
  File "/usr/lib64/python3.14/subprocess.py", line 578, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn · 2026-07-09T12:27:20Z

/image-refresh services

cockpituous · 2026-07-09T12:28:24Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://api.dev.testing-farm.io/v0.1/requests/c07b5271-4922-4d12-9b8e-4ce33c243a44

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "0243045f828923580527e9d70c7d2ef7e8c47cd0",
  "pull": 9260,
  "slug": "image-refresh-services-0243045f-20260709-122753",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-09T13:26:32Z

image-refresh services started on fbbacc122678. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-0243045f-20260709-122753/log.html

cockpituous · 2026-07-09T13:52:46Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

When testing there were stderr about memory issues for the freeipa container. Not evident by looking at the logs themselves but saw it without detached mode on. Tested using `bots/vm-run services -M 2000`. Might not work on your local machine as I also modified services image. See-also: cockpit-project/bots#9260 Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn · 2026-07-10T13:25:15Z

/image-refresh services

cockpituous · 2026-07-10T13:25:39Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/ad5dda82-4240-42c6-ae46-7d8bba107980

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "a58c084326286eb81b21ea629d5c568b48b15bdf",
  "pull": 9260,
  "slug": "image-refresh-services-a58c0843-20260710-132530",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-10T13:30:54Z

image-refresh services started on 89817ba351e5. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-a58c0843-20260710-132530/log.html

cockpituous · 2026-07-10T13:47:48Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

When testing there were stderr about memory issues for the freeipa container. Not evident by looking at the logs themselves but saw it without detached mode on. Tested using `bots/vm-run services -M 2000`. Might not work on your local machine as I also modified services image. See-also: cockpit-project/bots#9260 Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn · 2026-07-20T11:39:38Z

/image-refresh services

cockpituous · 2026-07-20T11:39:58Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/57edb6ac-28b5-4646-8b46-7572ca5823df

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "e7effa419d68588dd25991c8efddbc4880cd6262",
  "pull": 9260,
  "slug": "image-refresh-services-e7effa41-20260720-113950",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-07-20T11:46:30Z

image-refresh services started on c31eb1035a28. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-e7effa41-20260720-113950/log.html

cockpituous · 2026-07-20T12:02:41Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

This reverts commit 947a092.

Needed since last year when image was updated and had a breaking change with how they approach resolved. Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Using FreeIPA we started running into OOM issues. Lets bump it to 4GiB to make sure errors we see aren't related to that anymore. Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn · 2026-08-11T11:25:24Z

/image-refresh services

cockpituous · 2026-08-11T11:25:54Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/47f4830b-cd62-48cf-b468-9bc8ada345b1

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "38f05db1c121d0422abf835001859588b3b4a8a8",
  "pull": 9260,
  "slug": "image-refresh-services-38f05db1-20260811-112545",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-08-11T11:31:00Z

image-refresh services started on 38a550cbd7b1. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-38f05db1-20260811-112545/log.html

cockpituous · 2026-08-11T11:48:34Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn · 2026-08-11T11:51:06Z

/image-refresh services

cockpituous · 2026-08-11T11:51:28Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/be2525e4-fc06-4c30-bda4-dce9fb6dc8bf

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "38f05db1c121d0422abf835001859588b3b4a8a8",
  "pull": 9260,
  "slug": "image-refresh-services-38f05db1-20260811-115119",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-08-11T11:57:15Z

image-refresh services started on 6665f8fc5d38. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-38f05db1-20260811-115119/log.html

cockpituous · 2026-08-11T12:19:37Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn · 2026-08-11T12:41:21Z

/image-refresh services

cockpituous · 2026-08-11T12:41:50Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/753289f8-5df1-4481-9f86-ae5355b400ae

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "38f05db1c121d0422abf835001859588b3b4a8a8",
  "pull": 9260,
  "slug": "image-refresh-services-38f05db1-20260811-124141",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-08-11T12:47:59Z

image-refresh services started on e9199a9cbd03. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-38f05db1-20260811-124141/log.html

cockpituous · 2026-08-11T13:11:44Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Bumps from 5 min to 8 min for boot timeout Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn · 2026-08-11T13:38:56Z

/image-refresh services

cockpituous · 2026-08-11T13:39:20Z

Task scheduled: issue-9260 image-refresh/services

Testing Farm link: https://artifacts.osci.redhat.com/testing-farm/a6d39ea4-0fd2-4213-8fbe-bc17cd486df0

Job JSON

{
  "repo": "cockpit-project/bots",
  "sha": "3cafe2050da9ad3ff7fa930abe1633d0c92c682a",
  "pull": 9260,
  "slug": "image-refresh-services-3cafe205-20260811-133911",
  "context": "image-refresh/services",
  "command": [
    "./image-refresh",
    "--verbose",
    "--issue=9260",
    "services"
  ],
  "secrets": [
    "github-token",
    "image-upload"
  ]
}

cockpituous · 2026-08-11T13:46:58Z

image-refresh services started on fcd05494b323. Log: https://cockpit-ci-logs.s3.us-east-1.amazonaws.com/image-refresh-services-3cafe205-20260811-133911/log.html

cockpituous · 2026-08-11T14:05:30Z

image-refresh services failed: Command '('./image-create', '--verbose', 'services')' returned non-zero exit status 1.

Venefilyn added the needswork label Jul 8, 2026

Venefilyn mentioned this pull request Jul 8, 2026

test/realms: Bump memory to 2000MB cockpit-project/cockpit#23499

Merged

Venefilyn removed the needswork label Jul 9, 2026

Venefilyn force-pushed the revert/c9s-freeipa branch from 1d15d55 to 0243045 Compare July 9, 2026 12:27

Venefilyn requested a review from allisonkarlitskaya July 10, 2026 11:46

Venefilyn force-pushed the revert/c9s-freeipa branch from f0ef293 to a58c084 Compare July 10, 2026 13:25

Venefilyn force-pushed the revert/c9s-freeipa branch from a58c084 to e7effa4 Compare July 20, 2026 11:39

allisonkarlitskaya removed their request for review August 11, 2026 11:21

Venefilyn added 3 commits August 11, 2026 13:25

Revert "images: Pin freeipa container to an older version"

dea1e18

This reverts commit 947a092.

images: Add DNS server to podman-run

590018c

Needed since last year when image was updated and had a breaking change with how they approach resolved. Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

image-create: Bump services memory

38f05db

Using FreeIPA we started running into OOM issues. Lets bump it to 4GiB to make sure errors we see aren't related to that anymore. Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn force-pushed the revert/c9s-freeipa branch from e7effa4 to 38f05db Compare August 11, 2026 11:25

image: Increase boot timeout

3cafe20

Bumps from 5 min to 8 min for boot timeout Signed-off-by: Freya Gustavsson <freya@venefilyn.se>

Venefilyn added the needswork label Aug 11, 2026

Uh oh!

Conversation

Venefilyn commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Venefilyn commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

Venefilyn commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

Venefilyn commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

Venefilyn commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

cockpituous commented Jul 8, 2026

Uh oh!

Venefilyn commented Jul 8, 2026

Uh oh!

Venefilyn commented Jul 9, 2026

Uh oh!

cockpituous commented Jul 9, 2026

Uh oh!

cockpituous commented Jul 9, 2026

Uh oh!

cockpituous commented Jul 9, 2026

Uh oh!

Venefilyn commented Jul 10, 2026

Uh oh!

cockpituous commented Jul 10, 2026

Uh oh!

cockpituous commented Jul 10, 2026

Uh oh!

cockpituous commented Jul 10, 2026

Uh oh!

Venefilyn commented Jul 20, 2026

Uh oh!

cockpituous commented Jul 20, 2026

Uh oh!

cockpituous commented Jul 20, 2026

Uh oh!

cockpituous commented Jul 20, 2026

Uh oh!

Venefilyn commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

Venefilyn commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

Venefilyn commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Uh oh!

cockpituous commented Aug 11, 2026

Venefilyn commented Jul 8, 2026 •

edited

Loading